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REMARKS 

The present application was filed on December 27, 2001. In a previous Office 
Action, the Examiner required election of claims. Subsequently, Applicants elected the 
claims of Invention I (claims 1-28) without traverse and canceled claims 29-38. In the 
present Response, Applicants cancel claim 2 and amend claims 1 and 14. Claims 1 and 3-28 
are therefore pending. The amendments are supported, e.g., by originally filed claims 2 and 
25; page 5, line 17 to page 6, line 2; page 11, line 21 to page 12, line 10; and page 14, line 4 
to page 18, line 6. 

In the present Office Action, the Examiner rejected claims 1-28 as being 
anticipated by Jain et al., U.S. Patent No. 5,915,250. 

In the present Response, Applicants cancel claim 2 and incorporate the subject 
matter of claim 2 into claim 1. In particular, claim 1 now recites the following: "deriving a 
set of perceptual semantic categories for representing important semantic cues in the human 
perception of images, where each semantic category is modeled through a combination of 
perceptual features that define the semantics of that category and that discriminate that 
category from other categories wherein the perr.entual features and their combinations are 
derived at least in P *rt through suhj ^W exneriments performed with human observers " 
(emphasis added). Original, unamended independent claim 25 also recites similar subject 
matter: "program instructions for processing a set of perceptual semantic categories for 
representing semantic cues related to the manner in which human observers perceive and 
organize images, the semantic categories . . . comprising a combination of perceptual features 
that define the semantics of a particular category and that discriminate that category from 
other categories, where the, nerceotu al features an d their combinations are derived through 
...h jer-.tive experim ent, performed with human observers" (emphasis added). 

Similar subject matter is also found in original independent claim 14. Claim 
14 originally recited the following: "said data processor operating in accordance with a 
stored program for determining the semantic meaning of images in accordance with asetof 
p^tual semantic cate gories that were previously derived from human observers and that 



S.N. 10/033,597 
Art Unit: 2624 



represent important semantic cues in the human perception of images" (emphasis added). To 
present this claim in a better form for consideration on appeal, Applicants have amended this 
claim as follows: "said data processor operating in accordance with a stored program for 
determining the semantic meaning of images in accordance with a set of perceptual semantic 
categories that were previously derived at least in part throug h sub jective experiment, 
performed withfr em human observers and that represent important semantic cues in the 
human perception of images" (emphasis added). In this manner, amended claim 14 reduces 
issues for appeal and parallels the subject matter in claims 1 and 25. 

It is noted that the previous Response specifically argued that dependent claim 
2 was patentable over Jain. As noted above, the subject matter of claim 2 has been 
incorporated through amendment into amended independent claim 1 and to amended 
independent claim 14. Similar subject matter already was included in originally filed, 
unamended independent claim 25. Furthermore, in the outstanding final Office Action, the 
Examiner specifically responded to the argument regarding claim 2: 

Regarding claim 2, the Applicant argues (top of page 14) that 
there is no disclosure in Jain that teaches "the perceptual features and their 
combinations are derived through subjective experiments pe rformed with 
human observers". The Examiner respectfully disagrees. Again as discussed 
the claim l's argument, the claim's language are broadly claimed and that the 
Examiner can give broadest reasonable interpretation without reading the 
specification into the claim. Furthermore, the specification also gives a 
limitless definition regarding perceptual features/semantic cues/categories (as 
discussed in claim 1). Therefore, Jain clearly teaches the perceptual features 
(such as color, texture, shape, pattern, and object that are visual to 
human/user) and their combinations (column 4, lines 1-45) are derived through 
subjective experiments (the process of obtaining similar properties) performed 
with human observers (column 4, lines 1-45). 
Outstanding Office Action, pages 2-3. Therefore the amendments to claim 1 and claim 14 
should not be surprising and should be allowed for purposes of appeal. 

Regarding the Examiner's rejection above to claim 2 (the subject matter of 
which now resides in claim 1), Jain at col. 3, line 59 to col. 4, line 57 states the following: 
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The above needs are satisfied by the present invention which is 
directed to a system and method for "content-based" image retrieval, a 
technique which explicitly manages image assets by directly representing to 
visual attributes. A visual information retrieval (VIR) Engine provides an open 
framework for building such a system. A visual feature is any property of an 
image that can be computed using computer-vision and image-processing 
techniques. Examples are hue, saturation, and intensity histograms; texture 
measures such as edge density, randomness, periodicity, and orientation, shape 
measures such as algebraic moments, turning angle histograms, and 
elongatedness. Some of these features are computed globally, i.e., over an 
entire image, and some are local, i.e., computed over a small region in the 
image The VIR Engine expresses visual features as image primitives . 
Primitives can be very general (such as color, shape, or texture) or quite 
domain specific (face recognition, cancer cell detection etcO- The .basic 
philosophy underlying this architecture is a transformation from the data-rich 
representation of explicit image pixels to a compact, semantic-rich 
representation of visually salient characteristics. In practice, the design of such 
primitives is non-trivial, and is driven by a number of conflicting real-world 
constraints (e.g., computation time vs. accuracy). The VIR Engine provides an 
open framework for developers to "plug-in" primitives to solve specific image 
management problems. 

Various types of visual queries are supported by the VIR 
Engine as follows: 

Query by image property, wherein a user specifies a property or 
attribute of the image, such as the arrangement of colors, or they may sketch 
an object and request the system to find images that contain similar properties. 
The Engine also allows the user to specify whether or not the location of the 
property in the image (e.g., blue at the bottom of the image or blue anywhere) 
is significant. 

Query by image similarity, wherein a user provides an entire 
image as a query target and the system finds images that are visually similar. 

Query refinement or systematic browsing. With any of the 
previous modes of query, the system produces some initial results. A browsing 
query is one that refines the query by either choosing an image from the 
previous result set, or by modifying the parameters of the original query in 
some way. The system in this situation reuses the previous results to generate 
refined results. 

An important concept in content-based retrieval is to determine 
how similar two pictures are to one another. The notion of similarity (versus 
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exact matching as in database systems) is appropriate for visual information 
because multiple pictures of the same scene will not necessarily "match, 
although they are identical in content. In the paradigm of content-based 
retrieval, pictures are not simply matched, but are ranked in order ol their 
similarity to the query image. Another benefit is that content extraction results 
in very high information compression. The content of an image file may be 
expressed in as little as several hundred bytes of memory, regardless of the 
original image size. As an image is inserted into a VIMSYS database, the 
system extracts the content in terms of generic image properties such as its 
color texture, shape and composition, and uses this information for all 
subsequent database operations. Except for display, the original image is not 
accessed. Naturally, the VIMSYS model also supports textual attributes as do 
all standard databases. 

Jain, col. 3, line 59 to col. 4, line 57. Nowhere in this cited text does it disclose the subject 
matter in independent claim 1 of "deriving a set of perceptual semantic categories for 
representing important semantic cues in the human perception of images, where each 
semantic category is modeled through a combination of perceptual features that define the 
semantics of that category and that discriminate that category from other categories.wherein 
the EerceBtua] features ™d their co mbinations are derived at least in part through subjective 
experiments nerform^ with human observers" (emphasis added). Nowhere in this cited text 
(or any other part of Jain) is there any disclosure or implication of subjective experiments 
performed for any reason with human observers. 

While the above-cited text of Jain does discuss a "user", it is clear that the user 
is inputting data for a query. See Jain, col. 4, lines 21-33. Again, there is no disclosure or 
implication of subjective experiments performed for any reason with human observers. 

As shown above, the Examiner asserts the following: "Therefore, Jain clearly 
teaches the perceptual features (such as color, texture, shape, pattern, and object that are 
visual to human/user) and their combinations (column 4, lines 1-45) are derived through 
subjective experiments (the process of obtaining similar properties) performed with human 
observers (column 4, lines 1-45)." However, Jain does not indicate that "perceptual features 
(such as color, texture, shape, pattern, and object that are visual to human/user) and their 
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combinations" are derived through subjected experiments performed with human observers. 
For instance, Jain states the following: 

A visual feature is any property of an image that can be 
computed using computer-vision and image-processing techniques. 

Examples are hue, saturation, and intensity histograms; texture measures such 
as edge density, randomness, periodicity, and orientation; shape measures such 
as algebraic moments, turning angle histograms, and elongatedness. Some ot 
these features are computed globally, i.e., over an entire image, and some are 
local i e computed over a small region in the image. The VIR Engine 
expresses visual features as image "primitives". Primitives can be very 
general (such as color, shape, or texture) or quite domain specific (face 
recognition, cancer cell detection, etc.). 
Jain, col. 3, line 63 to col. 4, line 9 (emphasis added). It appears that visual features are 
computed using computer-vision and image-processing techniques and there is no disclosure 
that such features derived at least in part through subjective experiments performed with 
human observers. 

Furthermore, Jain also states the following: 

The VIR Engine looks at the pixel data in the images, and 
analyzes the data with respect to visual attributes such as color, texture, 
shape and structure. These visual attributes are called "primitives , and the 
image characterization is built up from these. Images which have been 
analyzed may then be compared mathematically to determine their similarity 
value or "score" . Images are analyzed once, and the primitive data is then used 
for fast comparisons. 

Jain, col. 6, lines 23-3 1 (emphasis added). There is no disclosure that primitives are derived 
at least in part through subjective experiments performed with human observers, as generally 
recited in independent claim 1 . 

Jain further states, under the heading "Primitives", the following: 

Image objects have computable image properties or attributes 
that can be localized in the spatial domain (arrangement of color), the 
frequency domain (sharp edge fragments), or by statistical methods (random 
texture) These computed features are called primitives. Primitives are either 
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global, computed over an entire image, or local, computed over smaller 
regions of the image. For each generic image property such as color, texture, 
and shape, a number of primitives may be computed. Besides this conceptual 
definition of a primitive, the specific implementation may also be referred to 
as a primitive. For instance, the collection of functions to extract and compare 
an image attribute may be referred to as a primitive. 

Jain, col. 7, line 62 to col. 8, line 7. See also col. 12, line 10 to col. 13, line 22, discussing 
image analysis. It appears that primitives are computed and there is no disclosure that 
primitives are derived at least in part through subjective experiments performed with human 
observers, as generally recited in independent claim 1 . 

Jain also states the following: 

The purpose of the Extensible Engine is to provide to the 
application developer the flexibility of creating and adding custom-made 
primitives to the system. For example, a face-matching system might construct 
primitives called "LeftEye" and "RightEye", and provide an interface that 
compares faces based on the similarity of their eyes. 

Jain, col. 13, lines 42-44. Nonetheless, there is no disclosure in Jain that such primitives are 

derived through subjective experiments performed with human observers. Jain further states 

that a "primitive is a semantically meaningful feature of an image. Thus color, texture, and 

shape are all general image primitives." Jain, col. 13, lines 49-51. Again, there is no 

disclosure in Jain that primitives are derived at least in part through subjective experiments 

performed with human observers. 

Applicants can find no disclosure in Jain of at least the subject matter in 
independent claim 1 of "deriving a set of perceptual semantic categories for representing 
important semantic cues in the human perception of images, where each semantic category is 
modeled through a combination of perceptual features that define the semantics of that 
category and that discriminate that category from other categories, wherein the perceptual 
features and their combinations are deriv e d at least in part through subjective experiments 
performed with human observers " (emphasis added). Therefore, amended independent claim 
1 is patentable over Jain. 
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Amended independent claim 14 and original, unamended claim 25 recite 
similar subject matter to the subject matter in independent claim 1 . In particular, amended 
independent claim 14 recites "said data processor operating in accordance with a stored 
program for determining the semantic meaning of images in accordance with a set of 
perceptual semantic categories that were previously derived at least in part through subjective 
ex periments performed with human observers and that represent important semantic cues in 
the human perception of images". Claim 25 recites "comprising program instructions for 
processing a set of perceptual semantic categories for representing semantic cues related to 
the manner in which human observers perceive and organize images, the semantic categories 
being modeled using multidimensional scaling and hierarchical clustering techniques and 
comprising a combination of perceptual features that define the semantics of a particular 
category and that discriminate that category from other categories, where the perceptual 
features and their combinations are deri v ed through subjective experiments performed with 
human observers ". Therefore, based on the arguments given above with respect to 
independent claim 1, independent claims 16 and 25 are also patentable. 

Dependent claims 3-15, 17-24, and 26-28 are all allowable at least by virtue of 
their dependency from allowable independent claims. Thus, the individual merits of the 
dependent claims need not be discussed at this juncture. 

Conclusion 

It is respectfully submitted that the amendment in this paper presents rejected 
claims in better form for consideration on appeal and therefore should be admitted under 37 
C.F.R. §1.1 16(b)(2). It is further respectfully submitted that claims 1 and 3-28 are patentable 
over Jain and the 35 U.S.C. §102 rejection to claims 1 and 3-28 should be withdrawn. 
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